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Abstract 


This document is the final report for Herc-SAR Task 106 — AIMS Feature Development. Several 
features to the AIMS simulation system, AJMSsim (previously called ELVISS), have been added 
to support human factors experimentation. The report summarizes the work performed and makes 
recommendations for the next phase. Software was developed to add scenario generation 
capability to the existing AIMSsim experimental research platform at DRDC. Core tasks were 
completed in the expected amount of time and some unplanned capabilities, such as path planning 
for targets, were added to the system to support an experiment by DRDC Atlantic. 


Resumé 


Le présent document est le rapport final du projet de Herc-SAR Task 106 sur |’élaboration des 
fonctions AIMS. Plusieurs caractéristiques du systéme de simulation AIMS, A/MSsim 
(anciennement appelé ELVISS), ont été ajoutées pour appuyer la recherche sur les facteurs 
humains. Le rapport résume les travaux effectués et fait des recommandations pour la prochaine 
phase. Un logiciel a été mis au point pour ajouter une capacité de génération de scenarios a la 
plateforme expérimentale AIMSsim existante de RDDC. Les taches principales ont été 
complétées a l’intérieur de la période de temps prévue et quelques capacités imprévues, comme la 
planification de trajets pour les cibles, ont été ajoutées au systéme pour appuyer une expérience 
mencée par RDDC Atlantique. 
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Executive summary 


Introduction 


A multi-sensor surveillance system, the Advanced Integrated Multi-sensor Surveillance 
(AIMS) system, is being developed to increase the capability of Search and Rescue (SAR) 
and Maritime patrol. The AIMS system will enhance the capability of SAR particularly at 
night and in poor weather. Earlier versions of AIMS were the Airborne Laser Based 
Enhanced Detection and Observation System (ALBEDOS), and the Enhanced Low-Light 
Level Visible and InfraRed Surveillance System (ELVISS). The AIMS system advanced 
through the integration of four sensors into a single gimball. To ensure optimal 
performance the AIMS system requires an appropriate interface and controls, the design of 
which must realize the interaction between technological capability and operator 
performance. A research platform that simulates use of the airborne sensor interface and 
controls has been developed at Defence Research and Development Canada (DRDC) to 
support evaluation of interface design concepts and to address human performance issues 
related to operating the AIMS and similar electro-optical imaging systems. 


Results 


Several important features to the AIMS simulation system, AJMSsim (previously called 
ELVISS), have been added to support human factors experimentation and this document is 
the final report for Herc-SAR Task 106 — AIMS Feature Development. The report 
summarizes the work performed in the project and makes recommendations for the next 
phase. Deliverables for this task were the updated (interim) software executable, updated 
user and system manuals, updated (final) software executable and source code, as well as 
this final report. A full month of testing and fixing of the new functionality was completed 
by DRDC Atlantic. Subsequent to this, some new capabilities were added to the system, to 
support the pilot-testing experiment by DRDC Atlantic, and experiment scripts were 
created. 


Future Plans 


Recommendations for the next phase focus on improving the configurability of the display 
component, improving the Scenario Generation Environment (SGE) to synchronize it with 
the AJMSsim and expand its capabilities to support the extensive experimentation 
capabilities of AJMSsim, and improving the creation and maintainability of experiment 
scripts. 


Significance 


The experimental research platform at DRDC provides a means for ensuring that the user is 
an integral part of the design process and optimal design from the user’s perspective is 
obtained. As technology advances and systems, like the AIMS, become more complex for 
an operator to use, user-machine system design becomes more critical and challenging. The 
continued development and upgrade of the A/MSsim research platform provides the 
experimenter with an appropriate level of simulation detail to conduct human peformance 
analyses which in turn delivers up-to-date knowledge and advice on the design of sensor 
surveillance systems to the military stakeholder. 
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Sommaire 


Introduction 


Systeme multicapteur de surveillance, le AIMS est en cours de développement pour 
améliorer les capacités de recherche et sauvetage (SAR) et de la patrouille maritime. Le 
systéme AIMS optimisera les capacités de SAR plus particuli¢érement la nuit et dans de 
mauvaises conditions météorologiques. D’autres versions du systeme AIMS avaient déja 
été développées, soit le systéme laser aéroporté perfectionné de détection et d’ observation 
(ALBEDOS) et le systéme perfectionné de surveillance a intensification de lumieére 
visible et a infrarouge (ELVISS). Le systeéme AIMS est supérieur a ses prédécesseurs 
grace a |’intégration de quatre capteurs dans un seul cardan. Pour offrir un rendement 
optimal, le systéme AIMS exige une interface et des commandes appropriées, dont la 
capacité doit intégrer les capacités technologiques et le rendement de |’opérateur. Une 
plateforme de recherche qui simule l’utilisation et la commande de |’interface de capteur 
acroporté a été élaborée par Recherche et Développement pour la défense Canada 
(RDDC) afin d’appuyer |’évaluation des principes de conception et pour aborder les 
questions relatives au rendement humain [16 a lutilisation du systéme AIMS et de 
systémes d’imagerie électro-optique semblables. 


Reésulats 

Plusieurs importantes caractéristiques du systéme de simulation AIMS, AIMSsim 
(anciennement appelé ELVISS), ont été ajoutées pour appuyer la recherche sur les 
facteurs humains et le présent document est le rapport final pour le projet Herc SAR Task 
106 : Développement des fonctions AIMS. Le rapport résume les travaux effectués et fait 
des recommandations pour la prochaine phase. Les résultats attendus pour cette tache 
Gtaient le logiciel mis ἃ jour exécutable (provisoire), la mise ἃ jour du manuel de 
Putilisateur et du manuel du systéme, la mise a jour du logiciel exécutable (définitif) et le 
code source, ainsi que la rédaction du présent rapport final. RDDC Atlantique a consacré 
un mois aux essais et aux ajustements des nouvelles fonctions du systéme. Ensuite, de 
nouvelles capacités ont été ajoutées au systéme pour appuyer des essais pilotes par RDDC 
Atlantique, et des scripts d’expérimentation ont été crées. 


Portée 


La plateforme de recherche expérimentale ἃ RDDC a fourni des moyens pour s’assurer 
que l’utilisateur fait partie du processus de conception et que la perspective de ce dernier 
sur la conception optimale est connue. Tout comme la technologie, les systémes comme 
AIMS se perfectionnent et deviennent de plus en plus complexes ἃ utiliser pour un 
opérateur; la conception du systéme utilisateur-machine devient de plus en plus important 
et impose de nouveaux défis. Le développement continu et la mise a niveau de la 
plateforme de recherche A/JMSsim procurent a l’expérimentateur assez de détail sur la 
simulation pour effectuer des analyses sur le rendement humain qui, a leurs tours, 
fournissent aux intervenants militaires une connaissance actuelle et des recommandations 
sur la conception de systémes de capteurs de surveillance. 
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1 Introduction 


The objective of this Call-Up was to add several important features to the AIMS 
simulation system, AJMSsim (previously called ELVISS), to support Human Factors (HF) 
experiments at DRDC Atlantic. 


The main deliverables from this tasking are the updated software executable, source code, 
and associated system and user manuals for the A/MSsim software, and this report, which 
summarizes the work done in this project. 


1.1 Background 


A multi-sensor surveillance system, the Advanced Integrated Multi-sensor Surveillance 
(AIMS) system, is being developed to increase the capability of Search and Rescue (SAR) and 
Maritime patrol. The AIMS system will enhance the capability of SAR particularly at night and in 
poor weather. Earlier versions of AIMS were the Airborne Laser Based Enhanced Detection and 
Observation System (ALBEDOS), and the Enhanced Low-Light Level Visible and InfraRed 
Surveillance System (ELVISS). The AIMS system is advanced through the integration of two 
sensors into a single gimball and the capability of rapid mounting on a non-dedicated aircraft. As 
such, the system will support timely SAR response using available aircraft equipped with the 
most advanced search and surveillance technology. Potential platforms for AIMS include a 
proposed fixed-wing SAR (FWSAR) aircraft and the CP-140 aircraft. To ensure optimal 
performance the AIMS system requires an appropriate interface and controls, the design of which 
must realize the interaction between technological capability and operator performance. 


1.2 Objective 


The objective of this Call-Up was to add several important features to the AIMS 
simulation system, AJMSsim (previously called ELVISS), to support HF experiments at DRDC 
Atlantic. 


13 This Document 


This document is the final report for Herc-SAR Task 106: AIMS Feature Development. 
Section 2 summarizes the deliverables accompanying this report. Section 3 of this document 
summarizes the work performed on this tasking. Section 4 makes some recommendations for the 
next phase of work. 
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2 Deliverables 


All the deliverables have been burnt to a Compact Disk (CD), to be sent to DRDC 
Atlantic. The CD contains: 


e Application: 
o AIMSsim-2_2_1.tar.gz: binaries and experiments 
o AIMSdb-2_2_1.zip: visuals database 

e Source: 


o AIMS-2 2 1-SVN-Bak-Jul_06.zip: the Subversion database containing 
the code including a history of the code. The subversion application, 
freely downloadable from subversion.tigris.org, is required to access the 
source code inside this zip file. A Graphical User Interface (GUD) front- 
end for subversion, called TortoiseSVN (tortoiseSVN.tigris.org), makes 
this trivial. 


e Documentation: 
o AIMSsim_ Manual System_Jul-2006.doc 
o AIMSsim Manual System Jul-2006.pdf 
o AIMSsim Manual User Jul-2006.doc 
o AIMSsim_ Manual _User_Jul-2006.pdf 

e Final report: 


o Herc-SAR Task 106 Final Report.pdf 
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3 Summary of Work Performed 


This section summarizes the work performed under this tasking. The high level tasks 
from the Statement Of Work (SOW) are as follows: 


1. Project management, 
2. Add new features to AIMSsim, 
3. Support DRDC Atlantic in developing an experiment using the new capabilities 


Program management tasks, including the initial meeting were performed as planned. 
The activities from the two main technical tasks (2. and 3. above) are summarized in the 
following sections. The project work started in December of 2005 and ended in July of 2006. 


In this phase, DRDC Atlantic decided that the software should be renamed from ELVISS 
to AIMSsim, to bring it up to date with the Standing Offer related to the AIMS system. 


3.1 Add new features to AIM Ssim 


All features described in the original SOW were implemented, with the exception of #10, 
Change Search History Display. Other features, not anticipated in the original SOW, took 
precedence over this one. 


Feature development led to departures from the original design, such as: 


- It became apparent while adding target motion (feature #4) to the system, that having 
all terrain intersections computed by the display component would adversely affect 
the quality of the simulation. A solution was designed which decoupled the 
simulation component from the display component, while maintaining some 
coherence of algorithms in both components. This required extra time. 


- It was determined that target motion could greatly benefit from the path following 
functionality available to the aircraft. A solution was designed that vastly expands the 
path-planning and following capabilities of aircraft and targets alike, allowing for 
complex, dynamic scenarios. This required extra time. 


- The terrain database, taken from the previous version of the system, was found to be 
inadequate for DRDC Atlantic’s planned experiment, due to presence of a tree-top 
surface spanning the whole terrain. The only practical solution was to temporarily 
make the clamping of targets onto the ground use the highest available terrain at the 
coordinate of interest. This required patching the software, until the terrain database 
can be redesigned and re-implemented in a future phase of the project. 


- Updating the manuals required more effort than expected, due in part to various 
unplanned functions that had to be created or, in some cases, renamed to make the 
global naming more coherent and object-oriented, and also due to many capabilities 
and important concepts lacking any mention in the previous version of the manuals. 


- The experiment changed often over the course of the project: the path, the events, the 
measurements. The actual measurements to make were defined only at the very end 
of the project and luckily the system was able to support them with only minor 
modifications to the source code. This required, in some cases, undoing some of the 
previous work. 


Several of the planned tasks were completed in less time than expected, such that the core 
tasks (#1 to 11) completed by the end of February 2006. This left a month for DRDC Atlantic to 
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thoroughly test the system prototype at their site. The software prototype was delivered at the end 
of March 2006, as per the SOW. 


The remaining project time was used on the remaining tasks, on improving the target 
motion to use the path following capabilities until then available only to the aircraft, to ease the 
creation of DRDC Atlantic’s experiment, and finally on helping DRDC Atlantic complete the 
creation of the experiment scripts in time for the pilot test to take place at the end of May, 2006 
(this was not in the SOW), occupying about one person-month of work (see the next section). 


The pilot testing revealed only minor fixes were necessary, plus the serious terrain tree- 
top surface problem mentioned above. The client decided that patching the terrain clamping of 
targets was the best option, albeit temporary. 


In the end, the client had to extend the project by only 20 hours, and the schedule had to 
be extended from ending in May to ending in July. 


3.2 Support for experiment development 


At the time of writing of the SOW, the experiment was only very vaguely defined, just 
enough to know what features to add to AJMSsim. As soon as the experiment was more defined, it 
became apparent that some of the features discussed in the previous section would have to be 
extended. 


The final experiment is much more complex than was anticipated by either side, but is a 
very nice example of the power and robustness of this application. This extra complexity required 
more time for support in the development of the experiment scripts, not only due to the 
complexity per se, but because the extra complexity meant that DRDC Atlantic could not create 
as much of the experiment scripts as anticipated. 


Some of the characteristics of the experiment: 


- There are six types of experiments: aircraft loops around stationary target, target 
loops around stationary aircraft, and both loop around each other - each conducted 
under manual tracking conditions and under automated tracking conditions. 


- The aircraft moves along a scanning-type path, going by targets along the way. The 
operator cannot control the camera orientation during this time. 


- There are a large number of targets, positioned randomly on each side of the 
aircraft’s path, according to the desired loop size. 


- All participants in the experiment get the exact same aircraft path and target 
locations, yet the loop sizes and orientations are random within an experiment. 


- Targets are activated (become visible) only when the aircraft arrives near their 
predefined target location. 


- When arrived, the camera automatically aligns itself to where the target is expected 
to be, whenever the aircraft arrives at the target. 


- The Moving Map Display (MMD) flashes for two seconds to notify the operator that 
they now have control of the camera orientation. 


- The operator is forced to designate the visible target, then zoom out completely, 
before the experiment can proceed. 


- The auto-tracking will be automatically turned on as soon as the loop is started, for 
three out of the six types of experiments. 


4 DRDC Atlantic CR 2006-278 


Herc-SAR Task 106: Final Report AIMS Feature Development 


- The aircraft will then loop around the stationary target, or the target loop around the 
stationary aircraft, or both loop around either other. 


- After a time unknown to the operator, the target will change color, indicating to the 
operator that they must designate again. 


- Tracking measurements take place while looping: the angular distance between the 
target and the center of the sensor window is saved to a data file, along with other 
various events. The field of view is also saved. 


- If the operator does not designate a second time, before the loop has ended, the 
system properly moves on to the next stage of the experiment. 


- The loops can have random deviations around a circular shape. Some loops will be 
right-handed, some left. The operator can never tell what the size or direction the 
next loop will be in, but all operators have the same loops. 


- In test mode, the aircraft motion can also be paused and resumed, while tracking is 
not taking place. 


- Thanks to the breakdown and representation of the experiment as a Finite State 
Machine in the software and scripts, the experiment always has control over operator 
input and other events: 


startupWait 


START gagESSED 
ContZoom/aM/toOps.lua 


pntDone 
ContZoomExp/st@@MMDFlashing.lua 


͵΄ 
ΤΑΣ ΟΥΞΤΙΟΚ ΤΙ “PRESSED 


LuaUtttstautomeckToggle.lua 


OPERATION UPDATED autoAligningToTarget | 
ContZoomExp/#ertSegAircraft.lua 


END_OF_P/giig: 
ContZoomExemmynToTarget.lua 


= =END_OF Pil: AIRCRAFT 


οτος JOYSTICK _Tigie2_ PRESSED 
t AIRCRAFT ContZoomE¥pmlesignate.lua 


᾿ 
RaAircraft.lua designer aited 
' 


BUTTON_Tgig-PRESSED 
ContZoomEy gleMotion.lua BUTTON_TI RELEASED 


ContZoomEx gleMotion.lua 
pausedAfterContact +---~~~ 
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4 Recommendations for the next phase 


The following sections give some recommendations for the next phase, with 
explanations. Some of these recommendations incorporate information from discussions with 
DRDC Atlantic during the project. 


4.1 Source Code Status 


The current source tree was examined to determine the status of the existing source code. 


The level of consistency between the Scenario Generation Environment (SGE) and 
AIMSsim is too low. E.g. the SGE does not make use of any of the code structures and functions 
used by AJMSsim for path-planning, target visibility flag, etc. This makes the transfer of data 
from SGE to A/MSsim rather prone to errors as both systems make different assumptions. The 
SGE should be synchronized with 4/MSsim to make proper use of as many of the code structures 
as possible. 


4.2 SimC ontrol 


The path following was not designed, originally, for tacking a path plan onto the end of 
another path plan. Though the design did turn out to support this capability, the fillets between 
two path plans do not behave properly when the last path segment is small. This situation should 
be corrected. 


The logging functionality available to the scripts makes use of the very versatile logging 
library used in AJMSsim. Currently however there is no ability to create new log sinks and destroy 
log sinks, limiting the variety of files that can be created during an experiment. This capability 
should be added to the system to complete the logging feature. 


Currently, the experiment initialization script must specify the terrain as one massive 
geometry file. This limits the re-usability of scene components. E.g. in the most recent 
experiment, the tree cover was not desired, but couldn’t be removed from the terrain. If AJMSsim 
supported adding a terrain and attaching other objects to it, the tree cover could have been 
designed separately from the terrain and not included in the experiment. This would also allow 
for an experiment to choose which “obstacles” (buildings, bridges), or other geographical features 
(lakes, etc.) to include. 


The scripting engine could use some improvements to make the experiments easier to 
create and maintain. E.g. all exported functions should be in a table so they are easy to identify as 
exported functions. Also, classes and methods should be used instead of functions, for targets, 
aircraft, and path plans at least. 


4.3 SimDisplay 


Scripting capabilities should be added to the display side of the system, to allow dialog 
screens to be created at run-time, for tasks like asking the user some questions, showing 
messages, etc. This would also allow the display to be configured, e.g. to toggle/position/size the 
different displays available. 


The simDisplay uses dead reckoning to move targets on the terrain, when no new data is 
available from simControl for the new frame being displayed. The algorithm used for dead 
reckoning could be improved to lead to smoother motion. 


The messaging system between simControl and simDisplay should be improved to 
prevent messages from arriving out of sync with their associated data from shared memory. This 
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has not been a problem so far but the likelihood of a discrepancy increases with time. The sooner 
this is fixed, the less impact it will have on existing experiments. 


The overlay text is really difficult to see. Real LCD displays have a background color that 
occludes the background image under the text, or the text lies outside of the image (though some 
overlays have to be over the image, e.g. in Automatic Target Recognition). It would be useful to 
add a background to text, or make it opaque or bigger or different color etc., or at least allow such 
attributes to be changed in the display configuration script. 


4.4 Documentation 


The application users could benefit from having the user and system manuals available 
online via HTML. This format would allow for a better separation of concepts yet make the 
documentation more easily navigable. The user and system manuals should be re-created in a 
web-friendly format, and material related to the SGE should be moved to a separate web-friendly 
document. 


4.5 Scenario Generation Environment 


It is recommended that the SGE interface should be cleaned up: several tabs refer to 
parameters which are no-longer available (they have been replaced by several parameters, set via 
function calls in scripts). 


The SGE interface should support: 


e Path-planning: ability to create any number of path plan objects (usable by 
aircraft and targets), instead of just one flight plan (usable by aircraft only), and 
to create a path plan from several other path plans. 


e Simulation of path following: show a dot moving along a path plan to see how 
the waypoint fillet, speed and acceleration rate affect the trajectory and time 
taken to execute the path plan. 


e Diagrammatic creation of the Finite State Machine (FSM) of an experiment: this 
would allow the experiment to create a series of labeled boxes and annotated 
lines, as in the FSM representation of the previous section, and generate the 
associated skeleton Lua code (script for FSM, and empty script for each 
transition). An annotation could be selected to start a Lua editor (third party 
application) on the associated script. 


e Connection to simControl: a “simulation” mode could trigger the simulation in 
“test” mode, allowing for testing of the scenario created with the FSM editor and 
external Lua editor. This would start the display in a smaller window size, and 
display the events and scripts being run. 


4.6 SimInputs 


The hardware input should be made more generic: ability to support USB joysticks, and 
ability to configure the function of each input signal understood by simControl. 
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3 Summary of Report and Next Steps 


This report details the deliverables for the project, mainly documentation, application and 
source, as well as the work performed to create those deliverables, including some of the hurdles 
that had to be surmounted, and finally makes a dozen or so recommendations for the next phase 
of AIMSsim. 


The next step is for DRDC Atlantic to determine which, if any, of the recommendations 
should be implemented, and discuss with Greenley & Associates any other features or capabilities 
needed. 
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